Human-Agent Dialogues on Health Topics - An Evaluation Study

A common conversation between an older adult and a nurse about health-related issues includes topics such as troubles with sleep, reasons for walking around nighttime, pain conditions, etc. This dialogue emerges from the participating human’s lines of thinking, their roles, needs and motives, while switching between topics as the dialogue unfolds. This paper presents a dialogue system that enables a human to engage in a dialogue with a software agent to reason about health-related issues in a home environment. The purpose of this work is to conduct a pilot evaluation study of a prototype system for human-agent dialogues, which is built upon a set of semantic models and integrated in a web application designed for older adults. Focus of the study was to receive qualitative results regarding purpose and content of the agent-based dialogue system, and to evaluate a method for the agent to evaluate its behavior based on the human agent’s perception of appropriateness of moves. The participants include five therapists and 11 older adults. The results show users’ feedback on the purpose of dialogues and the appropriateness of dialogues presented to them during the interaction with the software agent.